Accepted Jun 12, 2018; Epub ahead of print Aug 8, 2018

J Rehabil Med 2018; 50: 00–00

Correspondence address: Elisabeth Ekstrand, Department of Health Sciences, Physiotherapy Research Group, Lund University, Box 157, SE-221 00 Lund, Sweden. E-mail: elisabeth.ekstrand@med.lu.se

After stroke, many persons perceive difficulties in resuming meaningful activities, social relations and being in control of their own lives. To address participation restrictions is therefore important in stroke rehabilitation. The Participation domain of the Stroke Impact Scale, SIS Participation, is commonly used to assess perceived restrictions in participation after stroke. The domain includes 8 items: impact of stroke on work; social activities; quiet recreations; active recreations; role as a family member and/or friend; religious or spiritual activities; ability of life control and ability to help others. This study shows that the SIS Participation domain is reliable and can be used to assess perceived participation after stroke.

Stroke is one of the main causes of disability worldwide (1). It often leads to a variety of symptoms, such as sensorimotor deficits, cognitive dysfunction and communication difficulties, and affects the ability to perform daily activities and participate in society (1). The consequences of stroke can be classified according to the International Classification of Functioning Disability and Health (ICF), which describes functioning and disability as a dynamic interaction between body functions and structures, activities and participation (2).

According to the ICF, participation is defined as involvement in life situations, which includes being able to take part in meaningful activities, social relations and being in control of one’s own life (2). Many persons after stroke perceive participation restrictions due to difficulties resuming previous social activities and roles. To assess and improve participation after stroke is therefore an important goal in rehabilitation (3).

Perceived participation after stroke is commonly assessed with the participation domain of the Stroke Impact Scale (SIS) 3.0 (4, 5). The SIS is a Rasch analysed comprehensive interview-based measure of self-perceived health after stroke. It includes 8 domains, 1 of which, SIS Participation, covers participation and can be used as a separate scale (4).

In order to evaluate changes in perceived participation a measure must be reliable. Two previous studies (6, 7) have evaluated the psychometric properties of the SIS 3.0 (current version) and found that the test–retest reliability is acceptable to high. However, these authors did not specifically investigate and discuss the reliability of the SIS Participation domain and did not comprehensively evaluate the test–retest reliability in terms of agreement and systematic and random disagreements (measurement variability).

The aim of this study was to perform a comprehensive evaluation of the test–retest reliability of the SIS Participation domain as a measure of perceived participation in persons with chronic stroke.

Forty-five community-dwelling persons with ischaemic or haemorrhagic stroke, who had been treated at the Skåne University Hospital in the southern part of Sweden, were recruited from April to December 2013. The participants had mild to moderate disability (i.e. 1 to 3 according to the modified Rankin Scale (8)) and were at least 6 months post-stroke. The exclusion criterion was an inability to understand and follow test instructions due to cognitive impairments or communication difficulties.

Prior to inclusion, information about the study was provided and the individuals gave their written informed consent to participate. The principles of the Declaration of Helsinki were followed and the study was approved by the Regional Ethical Review Board, Lund, Sweden (Dnr 2012/591).

The SIS Participation domain (SIS 3.0) (4) includes 8 items: impact of stroke on work; social activities; quiet recreations; active recreations; role as a family member and/or friend; religious or spiritual activities; ability to control one’s life; and ability to help others. The items are scored on a 5-point scale according to the person’s self-perceived restriction during the past 4 weeks, where 5 = limited none of the time; 4 = limited a little of the time; 3 = limited some of the time; 2 = limited most of the time; and 1 = limited all of the time. The mean for the items is calculated as a composite score and converted into an index scale ranging from 0 to 100. A higher score indicates a greater degree of perceived participation. The SIS has been translated by MAPI Research Institute into several languages (http://www.mapi-institute.com) including a Swedish version, which was used in this study.

Participants were interviewed about their perceived participation on 2 occasions (referred to as test occasion 1, T1, and test occasions 2, T2), 1 week apart. The interviews were performed under similar conditions according to the COSMIN standards (9): at the same place in a quiet and separate room of the hospital, by the same interviewer (first author) and at the same time of the day. The SIS Participation domain took approximately 5–10 min to complete.

Prior to the ratings of SIS Participation the participants had rated their perceived strength in both upper and lower extremities according to SIS domain 1. They were also asked about their walking ability, if they used walking aids (yes/no), if they perceived shoulder pain (yes/no) or fatigue (yes/no). Their level of disability was rated according to the modified Rankin Scale (8).

Descriptive statistics, such as frequencies and means and standard deviations (SD), were calculated for demographic data and clinical characteristics of the participants. The total scores and item scores of SIS Participation were presented as medians and quartiles (i.e. 25^th and 75^th percentiles).

The test–retest reliability of SIS Participation was evaluated for the total score (ranging from 0–100) with Kappa statistics (quadratic weights) to assess the proportion of agreement observed beyond the agreement expected by chance (10). The strength of the Kappa coefficient was interpreted as < 0.40 poor, 0.40–0.75 fair to good, and > 0.75 excellent (11).

The test–retest reliability of the item scores (ranging from 0 to 5) was evaluated with the percentage agreement (PA) and the Svensson rank-invariant method (12). The PA was used to evaluate the agreement as the proportion of participants who rated the same score or ±1 point (PA ≤1 point) at T1 and T2. The Svensson method was used to evaluate systematic and random disagreement. A systematic disagreement occurs when the ratings are systematically higher or lower at the retest occasion and can be due to a learning effect, change in behaviour or fatigue. Random disagreement is the variability that cannot be explained by the systematic disagreement. The systematic disagreement, according to the Svensson method, is evaluated as the relative position, RP. Possible RP values range from –1 to 1 and zero values indicate a lack of systematic disagreement. A positive RP value indicates that the participants had higher scores on T2 than on T1. The random disagreement is evaluated by the relative rank variance, RV. RV ranges from 0 to 1 and a value larger than zero indicates the presence of random disagreement. The higher the RV value the more dispersed are the test–retest ratings. The RP and RV values are calculated together with a 95% confidence interval (95% CI) and a statistically significant disagreement is indicated by a 95% CI not covering zero.

Descriptive data were analysed with the IBM SPSS Statistics version 23 (IBM Corporation, Armonk, NY, USA), the Kappa coefficients with the MedCalc version 15 (www.medcalc.org), and systematic and random disagreement with the Svensson rank-invariant method (www.oru.se/esi/svensson). Probability values less than 0.05 were considered statistically significant.

Data on the 45 participants (37 men; mean age 65 years and mean time from stroke onset 44 months) are presented in Table I. All participants had mild to moderate disability (modified Rankin Scale 1–3) and were able to walk independently, but approximately one-third of the subjects used a walking aid outdoors. Approximately half of the participants experienced fatigue. A majority lived together with another person and most were retired.

Table II presents the medians and quartiles of the SIS Participation total score and item scores of the 2 test occasions. The median for the total score was 78 at T1 and 75 at T2. The medians for the item scores ranged from 3 to 5 at both test occasions. The maximum score of 100 was rated by 4.4% of the participants at T1 and by 15.6% at T2.

The Kappa coefficient of the total score was 0.78 (95% CI 0.74–0.83).

Table III presents the test–retest reliability of the item scores. The PA ranged from 47% to 98% and the PA ≤1 point from 82% to 100%. According to the Svensson method, most items showed a positive RP, i.e. the participants rated higher scores at T2, but only 2 items ”Active recreations” and ”Role as a family member” showed a significant systematic disagreement. No random disagreements (RV) were found for any of the items.

This study shows that the Participation domain of the SIS is reliable; the total score had an excellent agreement according to the Kappa coefficient and the item scores had high PA ≤ 1 point (> 82%). No items, except 2, showed systematic disagreement and no items showed random disagreement according to the Svensson method.

The Kappa coefficient of 0.78 of the total scores of SIS Participation was excellent. This result is in agreement with previous studies reporting intraclass correlation coefficients (ICC) of 0.79–0.87 (6, 7). However, as these studies (6, 7) used parametric statistics to evaluate the total score for SIS Participation it is difficult to fully compare their results with those of the current study.

Reliability analyses of the item scores showed that the PA ranged from 47% to 69% for all items, except for the item “Spiritual or religious activities”. This item had an agreement of 98%, but as most participants did not engage in religious activities they rated the item as “limited none of the time”. The PA ≤ 1 point showed high agreement (> 82%) for all items, indicating that, in most cases, the rating differed at most by 1 point. In the analyses of disagreements according to the Svensson rank-invariant method, most items showed a positive RP, indicating that the participants rated higher scores at T2. Only 2 items, “Active recreations” and “Role as a family member”, showed significant systematic disagreements and no items showed any random disagreements. It is difficult to compare these results with previous reliability studies, as they did not evaluate systematic or random disagreements according to the Svensson method. However, the findings indicate that the SIS Participation domain is reliable even if some variability in ratings between 2 test occasions occurs.

Several factors might have influenced the participants’ ratings of SIS Participation. Firstly, the term ”participation” could be interpreted differently by different people. According to the ICF, participation is defined as involvement in life situations (2). However, that basically includes all aspects of being a human and there is no clear distinction between the ICF components participation and activity. It has therefore been suggested that the definition of participation should be limited to social participation, i.e. societal involvement and social roles (13). Secondly, SIS was developed in the USA by input from patients, caregivers and stroke experts (4), but it has no theoretical underpinning. Cultural differences in a person’s activity repertoire might influence the ratings and, in fact, few participants in our study engaged in spiritual or religious activities. Thirdly, the SIS participation domain is constructed so that less restriction in participation is rated as higher scores, which could have been confusing for participants. It can also be difficult to distinguish between response option 4 (limited a little of the time) and 3 (limited some of the time), which might have resulted in the varied scoring at T1 and T2 in the present study. Moreover, most participants used the higher scores (i.e. 3–5) of the SIS Participation scale. This might be expected, as they all had a mild to moderate disability and were in a chronic phase (i.e. at least 6 months post-stroke), and had probably made adaptations in their lives. The somewhat higher ratings at T2 could have been due to a behavioural effect as the participants were more familiar and confident with the questions and the scale at the second test occasion. One way to reduce the systematic variability of the SIS Participation domain might be to perform more than 1 baseline assessment.

Taken together, as participation is not distinctly defined it might be difficult to measure. The broad aspects of the items of SIS Participation domain and the configuration of the scale might have caused variations in the participants’ ratings. Further refinement of the scale is therefore suggested in order to reduce variability. It would be interesting to develop an outcome measure of participation by using a bottom-up method (14) based on the narrower construct of social participation.

A strength of the current study is that all participants were in a stable phase post-stroke and the procedure was conducted according to the COSMIN standards (9). Care was taken to standardize the test situation, and the SIS Participation domain was rated at the same location, at the same time of the day, and with the same time interval between test occasions. Furthermore, 45 participants were included, which can be considered sufficiently large when reliability of an outcome measure is evaluated (9, 15). Persons with major cognitive impairments or difficulties communicating were excluded, and more men than women volunteered to participate. Therefore, the results cannot be generalized to the entire stroke population.

In conclusion, this study has shown that the SIS Participation domain is reliable in persons with chronic stroke and mild to moderate disability and can be used to assess perceived participation in this population.

The authors have no conflicts of interest to declare.